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High Availability of Resources in Telecommunications Network using 
Synchronized Redundancy Mechanism 

Field of Invention 

The invention resides generally in the field of management of 
telecommunications network involving nodes consisting of routers, packet 
forwarding engines, switches, etc. In particular, it relates to hot standby 
redundancy mechanisms at a node and to realization of improved availability 
in network resources. 

Background of Invention 

Telecommunications networks employ a wide variety of redundancy 
mechanisms in order to provide high resource availability. Generally 
speaking, a redundancy mechanism involves an active device and one or 
more inactive devices (standbys) and it improves resource availability by 
switching over to activate an inactive device when an active device fails or is 
taken off service due to a variety of reasons. However, routing and bridging 
protocol devices are not designed to easily allow dual or synchronous 
standby architecture within routers and switches to provide high availability. 
TCP/IP (Transmission Control Protocol/Internet Protocol), MPLS (Multi- 
Protocol Label Switching) BGP (Border Gateway Protocol) and OSPF (Open 
Shortest Path First) are examples of such routing and bridging protocols and 
are presently widely in use throughout the telecommunications network. The 
difficulty in implementing redundancy mechanisms in such protocols stems 
from the fact that such protocols exchange messages over 
telecommunications paths or connections which span over a wide 
geographical areas. Routing information, including setup/teardown of 
connections, must be constantly updated with co-operation by many nodes 
spread over the network. The active device uses such information to control 
the connections among nodes. When activated, the inactive device must 
also have same information to maintain the proper operation of the network. 
It is desired that no connections are dropped by an activity switch. 
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One known redundancy technique for addressing process failures 
involves configuring two or more devices from different vendors in parallel. It 
is assumed that devices from different vendors are susceptible to different 
types of failures, such as those that might be caused by an offending 

5 message, a hardware fault, or a software fault. This type of redundancy 
technique is generally expensive due to the inherent cost of the multiple 
devices and because using equipment from. multiple vendors causes 
additional operation, support, network management, and training costs. 
Additionally, the technique requires additional messages to be exchanged to 

10 move through paths onto tandem devices, if any, thus increasing cost, 
complexity, and network traffic. 

Among the redundancy techniques, some keep the inactive module 
or device (control card) in cold standby and others keep the inactive control 
card in hot standby with the active card. In the cold standby, an activity 

15 switch (switch-over from an active card to an inactive or standby card) 

requires starting of the inactive card from cold. The starting of the standby 
card generally involves discovering routes, connections or paths, loading the 
routing information and re-establishing connections. These operations flood 
the network and also take a substantial amount of time, resulting in dropped 

20 connections or other undesirable effects. In the hot standby scheme, some 
or all of the protocol stacks are run by both the active and inactive modules 
concurrently for at least a certain required period of time so that they both 
hold same routing information and other necessary data. An activity switch 
does not involve discovery of routes, connections or paths before the take- 

25 over by the inactive card but the active and inactive cards must be 

synchronous with one another so that a hitless (faultless) switch-over can be 
performed. The synchronization between two cards in the execution of the 
protocol stacks is important. Running a protocol stack in identical active and 
inactive control cards does not necessarily ensure synchrony due to the fact 

30 that the active card may have additional tasks that the inactive card needs 
not perform. It is also possible that both cards are of different releases or of 
different venders. In such cases, the same protocol stacks may not be 
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executed synchronously and may result in different decisions to be made on 
a same stimulus. 

United States Patent Application No. 20020141429 published on 
October 3, 2002, Pegrum et al, is entitled "High Availability Packet 
5 Forwarding Apparatus and Method". Its abstract states: 

A high availability packet forwarding router for an internet protocol (IP) 
network, includes two control processors, one or more service 
termination cards (STCs) with forwarding information bases (FIBs), 
and a packet forwarding engine. The two processors run 
10 asynchronously in a master/standby relationship. Integrity of 

processes running on the control processors is monitored and the 
forwarding engine forwards packets according to a FIB maintained by 
an in-service one of the control processors. Hitless failover and 
hitless software upgrades are supported. 
15 This application does not address how the control processors are kept 

in the master/standby relationship, particularly with respect to RSVP, RSVP 
being part of MPLS protocol. It is silent about how the two controllers are 
kept in hot standby to provide control processor protection to a single LSP 
(Label Switched Path). 
20 Applicant's United States Patent Application No. 1 0/350,81 7 filed on 

Jan. 24, 2003 entitled "Method and Apparatus for Facilitating Routing 
Protocol Redundancy in a Network Element" by Kendall Harvey et al 
discloses a method and apparatus for synchronizing routing protocol 
information associated with a plurality of routing modules of a network 
25 element. The application describes cases of adding an additional routing 
module to a network element or switching over to a standby module. One 
disclosed embodiment facilitates synchronization of redundant routing 
modules in a network element. In that embodiment, lower layer protocol 
(e.g., TCP) and higher layer protocol (e.g., BGP) tasks of a first routing 
30 module are synchronized with respective lower layer protocol (e.g., TCP) and 
higher layer protocol (e.g., BGP) tasks of a second routing module. The first 
routing module and the second routing module are redundant routing 
modules within a network element. Protocol information (e.g., TCP packets, 
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BGP packets, etc) that is processed on the first routing module (e.g., an 
active one of a plurality of redundant routing modules) is similarly processed 
on the second routing module (e.g., an inactive one of the plurality of 
redundant routing modules). Accordingly, this arrangement can be called a 

5 hot standby scheme in which both modules are kept synchronous because 
they both run concurrently on all the received messages. 

The described technique allows an activity switch without adversely 
affecting service. For example, after such an activity switch, a newly active 
routing module (i.e., previously the inactive routing module) processes 

10 routing updates that would normally be received by a newly inactive routing 
module (i.e., previously the active routing module). Furthermore, the newly 
active routing module does not become out of date with respect to routing 
information maintained on other network elements. In this manner, the 
network is not burdened by a burst of updates in response to the activity 

15 switch. Limiting the burden of such a burst of updates eliminates "churn" in 
the routing tables of network elements, thus improving performance of the 
network. Significantly, service of existing routes is maintained, and change 
to existing routes, deletion of routes, and addition of routes can continue 
uninterrupted; the switchover is transparent to neighboring routers. By being 

20 transparent to neighboring routers, a technique disclosed herein need not 
require cooperation of neighboring routers to enable an activity switch. 
Accordingly, neighboring routers need not be made aware of such an activity 
switch, nor do they need to support protocol extensions to enable such an 
activity switch. 

25 As mentioned earlier, running the both modules concurrently will 

not necessarily result in same decisions to be made on the same 
stimulus. The invention ensures that the active and inactive modules 
stay synchronized. 

The invention will be described in connection with the problem of 

30 improving availability in MPLS networks, particularly to providing high 
availability on LSPs that are set up and maintained using RSVP-TE 
(Resource Reservation Protocol-Traffic Engineering). It should, however, be 
noted that though the description below deals primarily with MPLS 
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environment, the concept is equally applicable to any other network 
architectures that exhibit similar difficulties. 

Figure 1 illustrates the essence of the problems that can be 
encountered in the network under discussion. In the Figure, an MPLS 

5 network 10 is shown as an example but as mentioned earlier, discussion 
should apply to similar networks. Nodes 1 , 2 and 3 along the path of an LSP 
12 continually send and receive messages, e.g., status messages 14, 
concerning the status of the LSP. Node 2 is shown in more detail as 
comprising an active card 20 and an inactive card 22, each of which contains 

10 a control module including, among other blocks, a block containing an MPLS 
state machine. When an activity switch 24 occurs from an active control card 
to an inactive control card in cold standby, the MPLS state machine in the 
newly active card performs a complete MPLS restart. This causes dropped 
connections. 

15 Figure 2 shows the nature of the messaging used for LSPs between 

two nodes. Each messaging consists of exchanges of traffic e.g., query- 
response or instruction-acknowledgement. The setup and teardown 
messaging each happen only once for an LSP, whereas the refresh related 
messaging happens often. The problem with a direct implementation of hot 

20 standby between the control cards by using e.g., the protocol hooks, is that 
with so much messaging it is difficult to maintain synchronization between 
the cards. 

Summary of Invention 

25 In accordance with one aspect, the invention realizes 

improvement in resource availability in telecommunications networks, 
by employing a hot standby scheme of the redundancy technique. The 
invention keeps the active and inactive devices in synchrony by the use 
of synchronization messages between them. The synchronization 

30 messages are sent only when a connection status changes. 

In accordance with a further aspect, the invention permits to 
retain synchronization of the inactive MPLS block with the active one so 
that activity switch can be performed in a hitless manner. 
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In accordance with yet another aspect, the invention achieves a 
hitless (or substantially hitless) activity switch by keeping the inactive 
device in synchrony with the active device only when a connection 
status changes, by the use of decisions made by the active device. 
5 In a further aspect, the invention uses synchronization messages 

to update the inactive device whenever the status of a connection 
changes and the active device makes decisions based upon the change 
in the status. 

In one aspect, the invention resides in the field of MPLS networks 
10 and employs a hot standby scheme in the redundancy mechanism. The 
active device processes the routing information related to a label 
switched path and makes necessary decisions based on it. Whenever 
the status of the path changes, the active device additionally sends 
synchronization messages to the inactive device so that the latter 
15 updates its state machine and forwarding table with the routing 
information and decisions. 

In another aspect, the invention is the result of realization that it 
is difficult to maintain an inactive MPLS controller in hot standby mode 
in synchrony with an active controller. It has been realized that RSVP- 
20 TE refresh related messages can be filtered out and yet the inactive 
controller can be kept synchronized with the active using messages 
concerning changes of connection status. 

In a further aspect, the invention resides in a node of a 
telecommunications network, which node comprises redundant routing 
25 processor modules, one module being an active module and the other an 
inactive module. The invention is particular is directed to a method of 
maintaining the inactive module in hot standby mode with the active module. 
The method comprises steps of sending from the active module to the 
inactive module only a status affecting message which affects the status of a 
30 connection being held by the active module and updating the inactive module 
in response to the status affecting message so that the inactive module is 
maintained in hot standby with the active module. 
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In accordance with a yet another aspect, the invention resides in 
a node of a telecommunications network, which node comprises redundant 
MPLS processor modules, one module being an active module and the other 
an inactive module. The invention is directed to a method of maintaining the 

5 inactive module in hot standby mode with the active module. The method 
comprises a step of sending from the active module to the inactive module 
only a status affecting message which affects the status of an label switched 
connection being held by the active module. The method further comprises 
steps of updating the inactive module in response to the status affecting 

10 message so that the inactive module is maintained in hot standby with the 
active module, and performing an activity switch from the active module to 
the inactive module to reverse the roles of both modules so that the label 
switched connection is now held by the newly active module and the newly 
inactive module is maintained in hot standby. 

15 In accordance with a further aspect, the invention is directed to a 

telecommunications node comprising two routing control cards which are 
interchangeably maintained as active and inactive at any one time. Each of 
the control cards comprises a distributed processor for running protocols for 
managing a communications path and a message processor for sending 

20 synchronization messages to the other control card to update the distributed 
processor of the other control card. While being maintained active, the 
distributed processor processes all signaling messages concerning the 
communications path and generates the synchronization messages 
indicating new status when the status of the communications path has 

25 changed. While being maintained inactive, the distributed processor for 
receiving the synchronization messages from the other control card to stay 
synchronized with the distributed processor of the other control card. 

Brief Description of Drawings 

30 Figure 1 is a schematic illustration of a telecommunications 

network showing nodes through which an LSP has been established. 

Figure 2 is a diagram showing messaging mechanisms, RSVP- 
TE, of MPLS protocol. 
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Figure 3 is a view of a control card in accordance with one 
embodiment, showing a variety of processor blocks. 

Figure 4 is a schematic illustration of synchronization of MPLS 
state machines in the active and inactive devices, in accordance with 
one embodiment of the invention. 

Detailed Description of Embodiments of Invention 

The invention is the result of realization that the active and inactive 
cards do not have to be in total synchrony throughout the execution of the 
their protocols. In fact it is difficult to maintain both the active and inactive 
cards in total synchrony. They may not be identical in construction because 
they may be of different release of the same product or they may be from 
different vendors. It is required that the inactive card should contain an 
updated state machine and necessary routing information at the time of an 
activity switch. In the MPLS environment, at least the MPLS state machine 
within the inactive control card needs to be in hot standby with the active 
control card. According to this embodiment, a hot-standby mechanism is 
introduced to MPLS control. This function reduces, or eliminates, dropped 
connections that normally result from a switch-over in MPLS control. 

According to one embodiment of the invention, the active control card 
filters out all refresh related messaging, and sends only setup and teardown 
messaging to the inactive control card, which is sufficient to maintain the 
inactive card in hot-standby with the active card. That is, the active card only 
propagates RSVP-TE messages when there is a change in the state of an 
LSP, instead of propagating all protocol messaging. It should be noted that 
in RSVP-TE, the refresh related messages are repeated transmission of the 
original setup message. In other protocols, other forms of messages may be 
used for refreshing the LSP. Furthermore, whenever there is an LSP state 
change, the inactive control card updates its forwarding tables accordingly. 

Figure 3 illustrates one of the embodiments and it is a view of a 
control card for handling traffic e.g., MPLS traffic on LSP. In the Figure 3, the 
control card contains various processors, e.g., a control processor, a run time 
control processor, distributed layer 3 (DL3) processors, a centralized layer 2 
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(CL2) processor and an IP routing processor. The blocks are identified by 
the processor they contain. It should, however, be noted that the blocks also 
contain all the necessary modules and devices to perform designed functions 
with the identified processors. The centralized layer 2 processor controls 

5 layer 2 connections such as ATM, PPP (point-to-point) connections. The 
DL3-1 processor executes the layer 3 protocol state machine such as MPLS 
state machine. In this particular embodiment, a second DL3 processor DL3- 
2 is provided to also handle ATM protocols. IP Routing processor runs IP 
routing protocols and holds forwarding tables. Run Time Control (RTC) 

10 processor is dedicated to allocating resources. Control (CTL) processor 
coordinates all the operations of various processors. In accordance with one 
embodiment, there are two control cards, each of which contains the above 
modules. One control card is active and another inactive. DL3-1 of the 
active card generates synchronization messages as needed. The Message 

15 processor of the active card sends them to the inactive card to keep DL3-1 of 
the inactive card in synchrony with DL-1 of the active card. The DL3-1 of the 
inactive card is therefore updated from time to time by the active card with 
respect to the information needed for hitless activity switch. 

Figure 4 depicts schematically synchronization of the MPLS state 

20 machine in the DL3 processors of the active control card 60 and inactive 
control card 62. The figure schematically shows running of MPLS state 
machine 64 and 66 in DL3 processors of the active and inactive cards. The 
DL3 processor with MPLS state machine on the active card runs the required 
protocols to operate the network, including sending updated forwarding 

25 tables to the IP routing processor when an LSP is setup or taken down as 
shown by an arrow 68. At the same time, the active card sends a state 
machine synchronization message 70 to the inactive card only when a 
change of state of an LSP occurs, e.g., LSP setup or tear-down completed. 
The inactive card 62 updates its MPLS state machine and forwarding tables 

30 accordingly. After an activity switch, the role of control cards 60 and 62 
changes and newly activated card 62 performs the same functions as did the 
card 60 as indicated at 72. Instead of processing all the signaling messages 
as done in a typical hot standby mechanism, the inactive card does not 
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process signaling messages but only updates its state machine and routing 
table in accordance with the message received from the active card, thereby 
avoiding the problem of different decisions being made by two controllers. 
Synchronization of the state machines of the active and inactive cards 

5 eliminates or reduces the number of dropped connections during an activity 
switch-over of the control cards. 

The state machine synchronization message should contain the LSP 
information which includes resources (e.g. bandwidth, colour/preferences, 
explicit route), endpoint information (e.g. MPLS label, egress interface), 

10 FECs (forwarding table entries), and the LSP ID. 

Since the inactive control card is kept in hot-standby, its MPLS state 
machine is synchronized to that of the active control card. Therefore, when 
an activity switch occurs there is no disruption to the state of the LSPs. 
After the activity switch, the CTL processor starts an audit on the 

15 newly active DL3 processor. It does this by comparing information on 

connections to that stored in its database. The CTL stores LSP information if 
it is the source node of the LSP. Midpoint nodes of an LSP will clean-up their 
connections in response to refresh related messages. 



